Efficient Incremental Beam-Search Parsing With Generative And Discriminative Models

Extended Abstract: This talk will present several issues related to incre-mental (left-to-right) beam-search parsing of natural language using generative or discriminative models , either individually or in combination. The first part of the talk will provide background in incre-mental top-down and (selective) left-corner beam-search parsing algorithms, and in stochastic models for such derivation strategies. Next, the relative benefits and drawbacks of generative and discriminative models with respect to heuristic pruning and search will be discussed. A range of methods for using multiple models during incremental parsing will be detailed. Finally, we will discuss the potential for effective use of fast, finite-state processing, e.g. part-of-speech tagging, to reduce the parsing search space without accuracy loss. POS-tagging is shown to improve efficiency by as much as 20-25 percent with the same accuracy, largely due to the treatment of unknown words. In contrast, an 'islands-of-certainty' approach, which quickly annotates labeled bracketing over low-ambiguity word sequences, is shown to provide little or no efficiency gain over the existing beam-search. The basic parsing approach that will be described in this talk is stochastic incremental top-down parsing , using a beam-search to prune the search space. Grammar induction occurs from an annotated tree-bank, and non-local features are extracted from each derivation to enrich the stochastic model. Left-corner grammar and tree transforms can be applied to the treebank or the induced grammar, either fully or selectively , to change the derivation order while retaining the same underlying parsing algorithm. This approach has been shown to be accurate, relatively efficient , and robust using both generative and discrim-The key to effective beam-search parsing is comparability of analyses when the pruning is done. If two competing parses are at different points in their respective derivations, e.g. one is near the end of the derivation and another is near the beginning, then it will be difficult to evaluate which of the two is likely to result in a better parse. With a generative model, comparability can be accomplished by the use of a look-ahead statistic, which estimates the amount of probability mass required to extend a given derivation to include the word(s) in the look-ahead. Every step in the derivation decreases the probability of the derivation, but also takes the derivation one step closer to attaching to the look-ahead. For good parses, the look-ahead statistic should increase with each step of the derivation, ensuring a certain degree of comparability among …